The robust tagging of unrestricted text : the BNC experience

The production of annotated machine-readable corpora has been a central activity of the UCREL team at Lancaster University, led by Geoffrey Leech, since the early 1980s. This commenced with the annotation of the LoB (Lancaster-oslo/Bergen) corpus with part-ofspeech information over the period l98l-84 (Garside, Leech and Sampson 1987). The work has continued with corpora which introduced syntactic annotation at the constituent level (Leech and Garside l99l ) and at the level of anaphora (Fligelstone 1992; Garside 1993), with corpora marked with word sense and key semantic relationship information (Wilson and Rayson 1993), and with aligned multi-lingual corpora (McEnery et al 1994). Most recently it has resulted in the development of the British National Corpus (BNC), a corpus of one hundred million words of varied written and spoken texts annotated with part-of-speech information (Leech 1993). The BNC was constructed by a team of publishers (Oxford University Press, I-ongman, and Chambers Harrap), academic institutions (Lancaster University and Oxford University Computing Senrices) and the British Library, over the period l99l-94; Lancaster was responsible for the grammatical tagging of the corpus. The various types of annotation listed above have always been inserted in the text by a mixture of automatic and manual procedures, generally making use of some form of probabilistic technique to assign an analysis to a text at the appropriate linguistic level, and this can then be manually post-edited if neressary to achieve the desired level of accuracy. Because large quantities of text are involved, the post-editing will always involve some assistance by the computer to minimize the time taken to implement the analyst's decision, and to check the validity of each change. The results of the manual correction